Why Captions Have To Be on Time

Closed captioning dramatically improves deaf people's enjoyment of television shows, and appears to augment the auditory signal for people with some degree of hearing impairment. However, reports from people with mild to severe hearing loss suggest that when there is a delay between the audio track and the caption, perceivers are confused unless they turn down the volume. These effects have not yet been investigated experimentally. This study provides a preliminary investigation of the importance of synchronisation of captions with auditory-visual material for hearing-impaired people's enjoyment and comprehension of captioned television programs. Two participants were presented with audio-caption delays of 0, 1, 2, and 4 secs in an auditory-visual condition and an auditory-only condition. Both enjoyment and intelligibility diminished over lag times. In general enjoyment and intelligibility were higher in the auditory-visual than the auditory-only condition, however, for the more severely hearing impaired of the two participants, both enjoyment and intelligibility diminished at a faster rate over delay times for the auditory-visual than the auditory-only condition. Thus at long delays the presence of the visual signal appeared to be distracting. These results are discussed in terms of perceptual mechanisms and practical applications for captioning.